A storage appliance is a computer that provides file service relating to the organization of information on storage devices, such as disks. The file server (or filer) includes a storage operating system that implements a file system to logically organize the information as a hierarchical structure of directories and files on the disks. Each “on-disk” file may be implemented as a set of data structures, e.g., disk blocks, configured to store information. A directory, on the other hand, may be implemented as a specially formatted file in which information about other files and directories are stored.
A filer may be further configured to operate according to a client/server model of information delivery to thereby allow many clients to access files stored on a server, e.g., the filer. In this model, the client may comprise an application, such as a database application, executing on a computer that “connects” to the filer over a direct connection or computer network, such as a point-to-point link, shared local area network (LAN), wide area network (WAN), or virtual private network (VPN) implemented over a public network such as the Internet. Each client may request the services of the file system on the filer by issuing file system protocol messages (in the form of packets) to the filer over the network.
A common type of file system is a “write in-place” file system, an example of which is the conventional Berkeley fast file system. By “file system” it is meant generally a structuring of data and metadata on a storage device, such as disks, which permits reading/writing of data on those disks. In a write in-place file system, the locations of the data structures, such as inodes and data blocks, on disk are typically fixed. An inode is a data structure used to store information, such as metadata, about a file, whereas the data blocks are structures used to store the actual data for the file. The information contained in an inode may include, e.g., ownership of the file, access permission for the file, size of the file, file type and references to locations on disk of the data blocks for the file. The references to the locations of the file data are provided by pointers in the inode, which may further reference indirect blocks that, in turn, reference the data blocks, depending upon the quantity of data in the file. Changes to the inodes and data blocks are made “inplace” in accordance with the write in-place file system. If an update to a file extends the quantity of data for the file, an additional data block is allocated and the appropriate inode is updated to reference that data block.
Another type of file system is a write-anywhere file system that does not overwrite data on disks. If a data block on disk is retrieved (read) from disk into memory and “dirtied” with new data, the data block is stored (written) to a new location on disk to thereby optimize write performance. A write-anywhere file system may initially assume an optimal layout such that the data is substantially contiguously arranged on disks. The optimal disk layout results in efficient access operations, particularly for sequential read operations, directed to the disks. A particular example of a write-anywhere file system that is configured to operate on a filer is the Write Anywhere File Layout (WAFL™) file system available from Network Appliance, Inc. of Sunnyvale, Calif. The WAFL file system is implemented within a microkernel as part of the overall protocol stack of the filer and associated disk storage. This microkernel is supplied as part of Network Appliance's Data ONTAP™ software, residing on the filer, that processes file-service requests from network-attached clients.
As used herein, the term “storage operating system” generally refers to the computer-executable code operable on a computer that manages data access and may, in the case of a storage appliance, implement file system semantics, such as the Data ONTAP™ storage operating system that is implemented as a microkernel. The storage operating system can also be implemented as an application program operating over a general-purpose operating system, such as UNIX® or Windows NT®, or as a general-purpose operating system with configurable functionality, which is configured for storage applications as described herein.
Disk storage is typically implemented as one or more storage “volumes” that comprise physical storage disks, defining an overall logical arrangement of storage space. Currently available filer implementations can serve a large number of discrete volumes (150 or more, for example). Each volume is associated with its own file system and, for purposes hereof, volume and file system shall generally be used synonymously. The disks within a volume are typically organized as one or more groups of Redundant Array of Independent (or Inexpensive) Disks (RAID). RAID implementations enhance the reliability/integrity of data storage through the redundant writing of data “stripes” across a given number of physical disks in the RAID group, and the appropriate caching of parity information with respect to the striped data. In the example of a WAFL file system, a RAID 4 implementation is advantageously employed. This implementation specifically entails the striping of data across a group of disks, and separate parity caching within a selected disk of the RAID group. As described herein, a volume typically comprises at least one data disk and one associated parity disk (or possibly data/parity partitions in a single disk) arranged according to a RAID 4, or equivalent high-reliability, implementation.
In order to improve reliability and facilitate disaster recovery in the event of a failure of a filer, its associated disks or some portion of the storage infrastructure, it is common to “mirror” or replicate some or all of the underlying data and/or the file system that organizes the data. In one example, a mirror is established and stored at a remote site, making it more likely that recovery is possible in the event of a true disaster that may physically damage the main storage location or it's infrastructure (e.g. a flood, power outage, act of war, etc.). The mirror is updated at regular intervals, typically set by an administrator, in an effort to catch the most recent changes to the file system. Typically, the source system will create a snapshot, or persistent consistency point image (PCPI), of a volume to be mirrored. Snapshots, or PCPIs, described further below, are a point-in-time representation of the storage system, and more particularly, of the active file system, stored on a storage device (e.g., on disk) or in other persistent memory and having a name or other unique identifier that distinguishes it from other PCPIs taken at other points in time. The PCPI provides a point in time image to be mirrored to the destination system.
In certain installations, a primary system is mirrored to a secondary system which, in turn, is mirrored to a tertiary system. This mirroring to a second system to a third, etc. creates a “cascaded” chain of systems. As each of the systems may be mirroring to its backup on a different schedule, a need arises to assure, for example, that should the secondary system fail, the tertiary system may continue to generate coherent images of the file system from the primary. As such, the primary and tertiary systems need to maintain at least one PCPI in common. This need to maintain at least one PCPI in common to enable continued mirroring operations in the event of a failure or error condition may be complicated by differing backup schedules between, e.g., the primary and secondary or the secondary and tertiary systems. One option is to save each completed PCPI for each system for a predetermined period in order to provide a sufficient time frame for the next system in the cascaded system to obtain a mirrored image of the data. However, should any system within the cascaded system change its backup or mirroring schedule, the possibility exist for systems along the cascaded chain to lack a common PCPI, which may result in a potential loss of data as the storage systems would be unable to resume mirroring. Another noted problem of such an arrangement is that, if any system in the cascaded chain of systems performs backup mirroring at a high frequency, e.g., every 15 minutes, then that backing-up system may generate and store a multitude of PCPIs which serve to reduce available disks space.
Another noted disadvantage occurs when, for example, a dump program is utilized to transfer a PCPI stored on one of the cascaded systems to offline storage, such as tape, for archival and/or disaster recovery purposes. In such situations, a mirroring program may complete its mirroring operation and delete a PCPI while the tape dump program still requires access to the PCPI for backup purposes. If the PCPI is deleted before the tape dump completes, the tape image may be incomplete and/or corrupted.
More generally, multiple applications that depend on various PCPIs stored across a number of storage systems require a technique for communicating their dependencies to other applications. These dependencies include when an application has a dependency on a PCPI that spans across multiple storage systems when, for example, the PCPI has mirrored.